An elementary analysis of ridge regression with random design

In this note, we provide an elementary analysis of the prediction error ridge regression with random design. The proof is short and self-contained. particular, it bypasses use Rudelson’s deviation inequality for covariance matrices, through a combination exchangeability arguments, matrix perturbation operator convexity.